Generative artificial intelligence (GenAI)--a set of advanced technologies capable of generating texts, images, videos, programming codes, or arts from instructions via sounds or texts--has taken the society by storm and exerted wide-range influences on many aspects of the world economy (Baldassarre et al. 2023; Mannuru et al. 2023; Sætra 2023). Although it had been around for years, GenAI came to public prominence since the introduction of ChatGPT in November 2022, a chatbox able to generate answers, reasoning, and conversations at human level. Since its introduction, ChatGPT and similar large language models have quickly made their ways into the investment industry. One common use of ChatGPT for investment is to give trading recommendations directly from news about a company (such as news articles or corporate communications) (Lopez-Lira and Tang 2023). A less direct approach is to rely on similar pretrained language models such as BERT (Devlin et al. 2018) and OPT (Zhang et al. 2022) to generate a sentiment score for each company which is then used to make trading decisions.

Traditional approaches in offline reinforcement learning aim to learn the optimal policy that maximizes the cumulative reward, also known as return. However, as applications broaden, it becomes increasingly crucial to train agents that not only maximize the returns, but align the actual return with a specified target return, giving control over the agent's performance. Decision Transformer (DT) optimizes a policy that generates actions conditioned on the target return through supervised learning and is equipped with a mechanism to control the agent using the target return. Despite being designed to align the actual return with the target return, we have empirically identified a discrepancy between the actual return and the target return in DT. In this paper, we propose Return-Aligned Decision Transformer (RADT), designed to effectively align the actual return with the target return. Our model decouples returns from the conventional input sequence, which typically consists of returns, states, and actions, to enhance the relationships between returns and states, as well as returns and actions. Extensive experiments show that RADT reduces the discrepancies between the actual return and the target return of DT-based methods.